Repairing Concavities in ROC Curves
نویسندگان
چکیده
In this paper we investigate methods to detect and repair concavities in ROC curves by manipulating model predictions. The basic idea is that, if a point or a set of points lies below the line spanned by two other points in ROC space, we can use this information to repair the concavity. This effectively builds a hybrid model combining the two better models with an inversion of the poorer models; in the case of ranking classifiers, it means that certain intervals of the scores are identified as unreliable and candidates for inversion. We report very encouraging results on 23 UCI data sets, particularly for naive Bayes where the use of two validation folds yielded significant improvements on more than half of them, with only one loss.
منابع مشابه
Repairing Concavities in ROC Curves
Declaration This dissertation is submitted to the University of Bristol in accordance with the requirements of the degree of Bachelor of Science in the Faculty of Engineering. It has not been submitted for any other degree or diploma of any examining body. Except where specifically acknowledged, it is all the work of the Author. 3 ABSTRACT Machine Learning applications require learning algorith...
متن کاملShape recognition: convexities, concavities and things in between
Previous studies on shape recognition have drawn different conclusions regarding the importance of specific object features, such as convexities, concavities and intermediate points. Some studies found evidence for a predominant role of convexities, whereas others favored concavities or intermediate parts. However, most studies have employed familiar objects or simple geometric shapes not neces...
متن کاملROC curves and nonrandom data ∗ Jonathan Aaron
This paper shows that when a classifier is evaluated with nonrandom test data, ROC curves differ from the ROC curves that would be obtained with a random sample. To address this bias, this paper introduces a procedure for plotting ROC curves that are inferred from nonrandom test data. I provide simulations and an example with wine data to illustrate the procedure as well as the magnitude of bia...
متن کاملSemiparametric Inferential Procedures for Comparing Multivariate Roc Curves with Interaction Terms
Multivariate ROC curve models that include an interaction term between biomarker type and false positive rate are important in comparative biomarker studies, because such interaction allows ROC curves of different biomarkers to cross each other. However, there has been limited work in drawing inference for comparing multivariate ROC curves, especially when interaction terms are present. In this...
متن کاملMixtures of receiver operating characteristic curves.
RATIONALE AND OBJECTIVES Receiver operating characteristic (ROC) curves are ubiquitous in the analysis of imaging metrics as markers of both diagnosis and prognosis. While empirical estimation of ROC curves remains the most popular method, there are several reasons to consider smooth estimates based on a parametric model. MATERIALS AND METHODS A mixture model is considered for modeling the di...
متن کامل